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ABSTRACT 

A project was designed to develop and test a library 
of cassette audiotapes for improving the technical skills of 
educational researchers. Fourteen outstanding researchers from 
diverse fields were identified, and a short instructional tape was 
prepared by each. Subjects of the tapes included instructional 
objectives for intellectual skills, sources^ of bias iri surveys, 
implications for the next 20 years of change, some precepts for 
conducting educational research, statistical interactions, evaluation 
skills, learning and instructional psychology, multiple regression 
analysis, control of student activity in learning, limitations "of : „ 
variables taken from common language, instant feedback displays of 
teaching behavior, research implications for reforms in educational 
finance, recent trends in the analysis of multidimensional 
contingency tables and nonparametric rank analysis of variance, and 
problems of conducting research with educable mentally retarded 
children. A separate evaluation of__how useful the tapes were to 
purchasers showed that those who listened to the tapes felt they 
increased their understanding of the topic discussed. (SH) 
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AUTHOR 1 S ABSTRACT 



This project was designed to produce a series of half hour 
cassette audiotapes to be used as. instructional materials to 
improve the skills of educational researchers. In addition to 
developing the cassette tapes, the project was to assess, at 
least initially, the degree to which such tapes could serve as 
an effective vehicle for the continuing education of such re- 
search personnel. To accomplish this the cassette tapes devel- 
oped during the project were turned over to the American Educa- 
tional Research Association (AERA) in order that they could be - 
distributed commercially by AERA, thereby testing the economic 
feasibility of developing and distributing such tapes. 

\ 

In all, 14 separate tapes were prepared by some of the 
nation's leading educational researchers, thereby totally more 
than seven hours of instruction. The first 10 of these were 
distributed by AERA during 1971, with more than 1,400 tapes 
being sold in less than eight months. It is apparent from these 
sales figures that educational researchers will, indeed, purchase 
such, tapes in sufficient quantities to make them economically 
feasible to produce. ' 

An external evaluation of the purchaser's satisfaction with 
the first ten tapes was conducted independently by Professor 
Jerry L. Brown of Indiana University. In general, Professor 
Brown concluded that the tapes were judged useful by their pur- 
chasers and that most purchasers reported a willingness to acquire 
more tapes. A number of procedural questions regarding usage of 
the tapes were explored as a consequence of responses to a ques- 
tionnaire distributed to tape purchasers. 
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INTRODUCTION 



In the present period of increasing specialization and rapidly 
expanding technical knowledge/ it is not surprising that profes- 
sionals in all fields have taken a serious look at the vehicles 
whereby their technical skills can be rennovated. "Continuing . 
education" for specialists of all kinds, e.g. , physicans , psycho- 
logists, and engineers, is a common theme in professional circles 
today. The project described in this report was initiated as 
a ccxisequence of the concern of educational researchers regarding 
appropriate methods for bringing recent technical advances to 
the attention of members of their profession, .itore specifically, 
the project tested the viability of using a library of cassette 
audiotapes, distributed by a national research organization, as 
a vehicle for improving the technical skills of educational 
researchers . 



The project emerged from the deliberations of a task force 
of the American Educational Research Association (AERA) commis- 
sioned to examine the training of educational researchers. The 
task^ force, appointed during 1969 under the chairmanship of Robert 
Gagne, had explored a variety of schemes for upgrading the pro- 
fessional knowledge and skills of educational researchers. In - 
appraising alternative methods of providing such continuing educa- 
tion, the AERA task force examined one technique which has recently 
been employed by comparable professional groups, namely, the use 
of cassette instructional tapes. As a consequence, a proposal 
was submitted to the Research Training Branch of the U.S. Office 
of Education to support the development of a pilot library of 
cassette tapes which dealt with recent advances in the strategies 
and procedures of educational research. 

In brief, the rationale for the proposal was that if high 
quality instructional tapes could be assembled for distribution 
among the educational research community, members of that group . ' 
would utilize these tapes to upgrade their own competencies. The 
arguments for the use of cassette tapes for this type of contin- 
uing education are several. First, they are relatively short, 
self-contained instructional modules and can be used, therefore, 
in a variety of circumstances. For example, the possibility of 
using cassette tapes during otherwise "dead" times such as driving 
to and from work would seem to be a compelling argument for such 
tapes. In addition, the possibility of using cassette tapes as 
a change of pace during working hours or after work also presents 
itself. In other words, there seemed to be numerous a priori 
agruments in favor of cassette audiotapes as a technique for pro- 
viding educational researchers with continuing education. What 
was needed was an empirical test of the merits of this approach. 

The particular scheme proposed in the project was to identify 
outstanding educational researchers and have each of them prepare 
a short instructional tape which would constitute an initial tape 
library. These tapes were to be distributed by the American 
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Educational Research Association at a price competitive v/ith those 
typically used commercial agencies distributing such tapes. If 
the tapes, were then purchased in sufficient quantities to make the 
continued development of such tapes economically self-supporting, 
organizations such as AERA could undertake their own tape-devel- 
opment efforts without federal support. 

The particular strategy involved in this project took advan- 
tage of the fact that many people in a field wish not only to 
learn of recent advances in that fitild but, perhaps, might 
derive some special dividends from listening to the, stature 
figures of their field dispense these remarks. In other words, 
it seems that if some individuals have an opportunity to read a 
paper by Robert Gagne or to hear remarks recorded by Gagr /9 him- 
self, some who would eschew the written document might find the 
recorded presentation sufficiently attractive to warrant its use. 

A recently reported survey* of the interest of American 
psychologists seems particularly relevant to the present investi- 
gation. In an effort to identify the interests of psychologists 
in using audiotapes a ten per cent random sample of the American 
Psychological Association's (APA) Division 12 (Clinical Psychology) 
and a five per cent sample of the general APA membership (exclud- 
ing Division 12 members) was administered a mailed questionnaire 
regarding the use of audiotapes and other audiovisual media. A 
return rate of 42 per cent from the general APA sample (n = 725) 
and a 49 per cent from the Division 12 sample (n = 255) was 
obtained. An extremely positive response regarding the potential 
utility of audiotapes was secured from both samples as reflected 
in the following findings: 



Eighty percent of the respondents from each 
of two samples of psychologists -7- a random 
sample of the general APA membership and a 
sampl t e from Division 12 (Division of Clinical 
Psychology) — said that they would use audio- 
tapes if these were available in an area of their 
interest. A little under 8 C% of the ^respon dents 
indicated that audiotapes have advantages over 
such traditional media as books and journals. 

The advantages most frequently mentioned were that 
tapes can be used more effectively in classrooms 
and other groups, they provide a sense of the 
person or persons communicating as opposed to the 
disembodied hand of the author, are convenient to 
use (in a variety of environments, according to 
one's own personal schedule), and present certain 
types of information (e.g. , dialogues and thera- 



*Prescott, Suzanne, Psychologist ' s Interest in Audiotapes and 
Other Audiovisual Media: Survey Results , National Information 

;;ys tern for Psychology/American Psychological Association, Washington, 
D.C., October, 1970. 
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peutic sessions) more effectively. An extrapolation 
of the survey results to the entire APA membership 
indicates that approximately 10,000 members would • 
use them if audiotapes weu easily available. Of 
course, many of the respondents (between 50% and 
60%) reported that they have already had experience 
with audiotapes. 



Respondents in the study also indicated a tangible interest in an 
audiotape cassette program. When asked if they would be willing 
to purchase tapes at prices ranging from $5.00 to $15.00 for / 
half hour or one hour cassette tapes, over 55 per cent of botlf 
sample's respondents reported that they would buy tapes at x tViese 
prices . » Thus, it appears quite clear from the APA survey- Chat 
psychologists appear highly positive toward the potential utility 
of audiotapes as a vehicle for their- continuing education. 

Turning to the present investigation, the funds requested 
from the U.S. Office' of Education were $14, 800 ,/With $1,000 
designated for the external evaluation report./ Because the 
attempt was to secure ten half-hour c ass ette^t apes for this price, 
the project can be designated as a low cost: development effort. 

In other words, in contrast to heavily financed efforts to develop 
instructional materials through the usfe of empirically-based, 
trial-revision sequences, this project was clearly at the other 
end of the cost continum. It was , /candidly , an effort to secure 
the services of several ^pf America's most able educational researchers 
in connection with an instructional mission, namely, an effort to 
upgrade skills of their fell>ow researchers. 

• /' / 'i 

Perhaps the most realistic way of describing the project will 
be to trace chronologically the key activities associated with the 
enterprise during its 18 months duration. Accordingly, the next 
section of the report will document these activities in a temporal 
sequence. At the'conclusion of the description of the project, a 
separate external evaluation report by Professor Jerry Brown of 
Indiana University will be presented. Following that evaluation ’ 
report, appendices will conclude the final report. 

/ - 



DEVELOPMENT PROCEDURES 



v June, 1970 . The project was officially approved in June, 1970 

and, as with most such projects, the serious thinking really got 
y under way regarding how to carry out the enterprise after official 

approval. The project was, in essence, to be conducted from the 
West Coast because of the project director's affiliation with the 
University of California, Los Angeles. At the same time it was 
to be fiscally managed from Washington, D.C. through the central 
office of the American Educational Research Association. Accord- 
ingly, preliminary communications regarding the needed arrangements 
between the project director and the AERA central office were 



O 



3 



worked out ( largely by mail, during June. A rough sequence of 
events schedule was planned during June and preliminary conversa- 
tions were undertaken with technical service agencies which might 
be involved in_tbe~reQording and distribution of the cassette 
training tapesT ^ 



Julv ; 1970 . As indicated above, the AERA Task Force for 
the Training of Educational Researchers had been involved in the 
exploratory discussions which led to the submission of the pro- 
posed project to USOE. During those deliberations, members of 
the group had indicated a willingness to make suggestions regard- 
ing potential contributors to the initial series of tapes. Accord 
ingly , on\July 10, 1970, a memorandum was issued by the project 
director to members of the AERA Task Force soliciting their sugges 
tions regarding contributors to the cassette tape library project. 
Although it was indicated that the task force wpuld, at a later 
point, be givet^ an opportunity to vote on the rrfest appropriate 
contributors, this mid-July effort was simply designed to secure 
nominees. Members of the task force at that time were the follow- 
ing: Abbott L. Ferriss, Robert M. Gagne, William j. Gephart, 

Gary E. Hanna, Jphn E. Hopkins, Reginald L. Jones, Jason Millman, 
Harold E. Mitzel, Ellis Page, Sam D. Sieber, and Blaine R. Worthen 
Responses from the Task Force members were returned to the project 
director as early as late July. ‘ 



Also during the summer the project director consulted with a 
number of colleagues regarding names to include in the initial 
list of potential contributors. Officials of AERA were contacted. 
Colleagues from different universities were consulted. An effort 
was made to secure a balanced representation for the list of poten 
tial contributors. To illustrate;, there is a general tendency 
among educational researchers when considering continuing educa- 
tion to think most quickly, of those colleagues involved in method- 
olgoical specialties, e.g. , statistics or research design. We 
were hoping to secure a wider array of educational researchers, 
hence deliberately sought the counsel of people who were in such 
fields as economics of education, sociology of education, etc. 



August , 1970 . A letter was directed to the Divisional Vice- 
Presidents of AERA. These are individuals who have particular 
specialties in different fields and, consistent with our effort 
to secure a wide array of possible participants, it was believed 
that such individuals could offer additional names to be added 
to the- list. A self-addressed post card was included for all 
Divisions (at that time there were seven AERA Divisions) and all 
but one of the Divisional Vice-Presidents responded with names 
and possible topics for the cassette library. 

September , 1970 . With the returns of all but two members 
of the AERA Task Force, all but one of the AERA Divisional Vice- 
Presidents, and many colleagues, a list of fifty educational 
researchers along with their fields of specialization was pre- 
pared. This list was transmitted to members of the AERA Task 
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Force on September 3rd with a request that each of the task force 
members check ten of the names they thought most suitable for the 
first effort to assemble a library of cassette training N tapes . 

All but one of the Task Force members returned their lists after 
a single follow-up mailing. By early October these "ballots" 
were counted and the fifteen most frequently checked individuals 
wore designated as those who would be 'contacted for possible 
participation in the project, it was anticipated that for a 
certain number of these individuals there would be conflicts which 
would prevent their participation. As a consequence, although 
only ten tapes were to be prepared during the first round of pre- 
sentations, it was thought that fifteen invitations would give us 
a reasonable number of alternatives . 

At the same time, with the prospect of moving more rapidly 
toward the actual recording dates, a Los Angeles organization, 
American Tape Duplicators (ATD) , was contacted to discuss the 
technical requisites of the recording session, it had been origin- 
ally anticipated that two conferences would be held in Los Angeles 
in early Spring, one of which would serve as a vehicle for feed- 
back regarding each participant's presentation, the second. of 
which would be used for final recordings of comparable quality . 
American Tape Duplicator is an extremely large organization with 
considerable national distribution, hence we were confident that 
their advice would be sound. We were advised by their technical 
recording specialists that under no circumstances was it necessary 
to have a final recording session in Los Angeles. The ATD tech- 
nical personnel informed- us that the common practice in such pro- 
jects w£s to have individuals prepare final recordings at their 
home base and that, by the use of reasonably inexpensive cassette 
recorders with automatic volume control, recordings of thoroughly 
satisfactory quality could be secured. ATD officials strongly 
advised us to dispense with the final recording session in Los 
■ Angeles, since they indicated this would create an unnecessary 
eaqpense. 

In view of the ATD staff's recommendations, it seemed that we 
could save the money used for the second recording session and, 
perhaps, schedule a second round of recordings at a later date. 

As a consequence, it was concluded that only one meeting in Los 
Angeles would be held in the Spring of 1971. 

November , 1970 . Several weeks thereafter, a letter was trans- 
mitted to the fifteen individuals on our list. The letter, a copy 
of which is included in the appendix, described the nature of the 
training project, stressed the manner in which individuals were 
selected, and urged the participation of each candidate. A £500 
honorarium plus all expenses for the participation was described. 

December , 1970 . Responses from letters of invitation were 
returned sporadically during December. Of the fifteen letters of 
invitation, ten positive respones were secured. In all, the 
following individuals agreed to participate in the cassette tape 
project: Dr. Robert Gagn4, Florida State University; Dr. Donald 
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L. Meyer, Syracuse University; Dr. John R. Platt, University of 
Michigan; Dr. A. A. Lumsdaine, University of Washington; Dr. Jason 
Millman, Cornell University; Dr. Michael Scriven, University of 
California, Berkeley; Dr. Robert Glaser, University of Pittsburgh; 
Dr. Fred_N . Kerlinger,, New York University ; •Dr.. Ernst Rothkopf, 
Bell Telephone Laboratory;, and Dr. Robert M.WV, Travers, Western 
Michigan University. i ^ 

s' ’ • . \ . ■ 

J anuary , 1971 . During January details were worked out for 
an anticipated March meeting in Los Angelos and schedules of the 
ten participants were tested in order -to locate a two or three 
day meeting span'when all individuals could, attend. After several 
rounds of letters and postcards, March 5, 6, and 7 appeared to be 
a three day period when all of the participants could attend the 
meeting in Los Angeles. 



, 1 

After checking with several hotels near the International Air- 
port, one was selected which offered both reasonable room rates 
and a meeting room set-up necessary for the particular recording 
plan we were attempting to implement. 

Also during January two letters were sent to the participants 
reminding them that in working out their presentation "design they 
should definitely cleave to an instructional effort rather than 
simply presenting "their view of the world. " -""These tapes were 
suppose-i t.o improve the competencies of those listening to them 
and not pre /ide a platform for rambling discursioris on the part 
of the .t jer^enter. This point was thus emphasized dn two written 
occasions during January as well as stated in the initial invitation 
letter. 

; \ 

i ‘ 

February 1971 . In February invitations to attend the March 
5-7 session were administered to almost 200 Southern California 
educational researchers. In general, these invitations were issued 
via duplicated notices, and in some instances personal notes were 
attached. There was some concern on the part of the director that 
too many individuals would attend the early March recording sessions 
As originally planned in the proposal submitted to the U.S. Office 
of Education, the March meeting was set as a time for feedback 
from individuals who had listened to the ten participants' half 
hour presentations. The presenter was then to interact with the 
audience and, on the basis of audience suggestions, prepare a re- 
vised version which would be the final tape included in the cassette 
series. As a consequence, we needed a reasonably representative 
audience for our March session. ■ February was spent in trying to 
locate such an audience. 



March, 1971 . The initial feedback session was scheduled, as 1 

indicated, on March 5-7, at the Los Angeles International Hotel. 

The following schedule was developed for the meeting. 



Date 
March 5 




March 6 



March 7 



O 




X 



AERA CASSETTE TAPE LIBRARY PROJECT 





March 5, 6, and 7' 




Time 


Presenters 


Title 


8:30 


Dr. Robert Gagn£ 

Florida State University 


Defining Objectives for Six 
Varieties of Learning 


10: 30 


\ 

Dr. Donald L. Meyer 
Syracuse University • 


Non-sampling Errors in 
Survey's and Experiments 


Lunch 

1:30 


Dr. John R. Platt 
- University of Michigan 


The Next Twenty Years 
of Change . 


8:30 

/ 

10 : 30 

/ 


Dr. A. A. Lumsdaine 
University of Washington 


Some Precepts for Conduct ’ 


Dr. Jason Millman 
Cornell University 


Statistical . Interactions 
-Their Nature and Importance 


Lunch 

/ 


1 




/ 

1:30 


Dr. Michael Sc riven 
University of California 


Evaluation Skills ' 

\ 


8:30 


Dr. Robert Glaser 
University of Pittsburgh 


\ 

The Psychology of Learning v 
and Instructional Psychology 


10: 30 


Dr. Fred N. Kerlinger 
New York University 


Multiple Regression Analysis 
and Educational-’- Research 


Lunch 




\ \ 


1:30 


Dr. Ernst Rothkopf 
Bell Telephone Lab. 

t 


Control of Student Activity 
in Learning 


3:30 


Dr. Robert M.W* Travers 


The Limitations of Variables 
Derived from Common Language 
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Th<* clones: a 1 plan tor the meeting was as follows: A presenter 

-,ih) taker* to a separate room where he delivered his presentation 
with only an audio technician present. His remarks were/ Lrans- ~ 
m:t*od to - a second room where an audience listened to the presen- 



i. at. 
i a : 
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on. a recording was made of this initial presentation, both 
presenter's purposes in revising his tape as well as for 



safety purposes (if the presenter failed to deliver on his promised 
revision of the tape, it was. always possible that „we could use the 

or.icnnal ) . 



The use ot separate rooms was considered imperative to create 
a condition whereby the* speaker would have to rely solely on an 
audio corrcnun icvition •'medium and would not be able, to employ the 

typical facial expressions or gestures which accompany an in- 

person presentation. We were anxious to have our speakers try 
to reach an unseen audience, for this is precisely what - they 
would have to do on the final version of the cassette tape they 
were to subsequently record. 



One hour was scheduled for the post-presentation interaction 
of a participant with the audience. These discussions- were to be 
scheduled around two central foci, first, a consideration of the 
n t a t i on techniques employed by the speaker and, \ second, a 
consideration of the substance of the topic of the presentation. 

We hoped that from both types of considerations perceptions re- 
garding desirable changes might emerge. A stenographer was to take 
notes of the one hour interaction session and summarize these for 
use by the presenter.* 

% ’ ... 

The March meeting, other than the problem^ with the steno- 
graphic feedback system, went off without a hitch. The presenters 
arrived on schedule and many participated in reacting to other 
presenter's presentations. These discussions were lively and 
extremely stimulating. One disappointment experienced by the direc- 
tor was the fact that very few of the two hundred people invited 
attended the meeting. In some cases there was an embarrassingly 
small turnout, for a given session/ e.g., 8-10 people. One can 
always debate whether a small group of^people reacting to a pres- 
entation yield as useful information as a large group. Certainly, /' 
there were some suggestions garnered from the sessions. But there 
were surely fewer people in the audience than had been anticipated. 



•It. should be pointed out that this planned feature of the March- 
session proved to be a failure. In spite of repeated reassarance 
from the local stenographic .firm' (located only after considerable 
difficulty), the night before the meeting was scheduled to take 
place the firrfi called and indicated there would be no stenographer 
present. We were fortunate in persuading a female colleague in the 
UCLA Department of Education who had stenographic experience, to pincl 
hit in the situation, but the press of her other responsibilities 
delayed the immediacy of the feedback to the participants. We had 
hop£d to have transcribed notes in the hands of the participants 
b.4 ore they left Los Ange’ *s or at the. very latest, one or two days 
after their return to their home base. As it worked out, however, 
in some cases the participants did not receive these summarized 
suggestions until almost two weeks after the presentation. 



/ 







\ 





April ju^d May , 197.1. The participants then returned to their 
home bases where a cassette recorder plus blank cartridge. had been 

them along with .instructions as to how—the"' final I'ecbrding 

v L as to be prepared. Most of the presenters returned their final 
tape within the 'suggested two or three weeks, but in a fev/ cases 
it took letters, phone calls, telegrams, and threats of guerrilla 
warfare to get the final version of the tape. The last tape 
finally was delivered/ in late May. 

Publicity inf ormation was released by the American Educational 
Research Association indicating that the tapes would be available 
for April distribution • In reality they were initially distributed 
in May. The price set for the individual tape was $8.00 apiece 
for non-AERA members, $7.00 apiece for AERA members, with a discount 
if the entire series of .ten were purchased, i.e., $70.00 for non^- 
members and $60 # 00 for members. 



. # 

The ten cassette tapes resulting from the March 5-7 Los Angeles 

meeting are briefly described below: | 

1 

1. Robert M. Gagnfc, "Defining Objectives for Six Varieties 

of Learning." 

' t 

A new method of describing instructional objectives 
for six varieties of intellectual skills. Distinc- 
tions are. drawn between the primary, and secondary 
verbs which are employed in writing instructional 
objectives and operational meanings for six primary 
verbs are presented. 

2. Donald L. Meyer, "Non-sampling Errors in Surveys and 

/ ' Experiments . " ■' 

• - * ’• i . 

. Sources of bias in surveys are discussed,. together with 

a model for incorporating assessments of errors into 
the -final reported estimate. The listener derives 
an insight into the basic ingredients of Beyesiah 
statistical methods. 

3. John R. Platt, "The Next Twenty Years of Change. " | 

Professor Platt observes that we are in the midst \ 
of a great world transformation, like ten Indus- \ ■ 

trial Revolutions and Protestant Reformations 
rolled into one, and it 1 ' will change education 
completely in the next few years.. There will 
be early enrichment for the young, continuing \ 

education and retraining throughout the rest of 
x x life, more sophisticated grade schools and colleges, 

and world education of billions of children by 
satellite broadcasts, radio and television. Impli- 
cations of these changes are examined. 

\ 

s 
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4. A. A. Lumsdaine, "Some Precepts for Conduct." 

General- considerations in conducting educational 
research are examined by Professor Lumsdaine, par- 
ticularly those attitudinal bents which lead to 
worthwhile research endeavors. 

5. Jason Millman, "Statistical Interactions: Their Nature 

and Importance . " 

Through the use of examples taken from a wide 
variety of educational research settings, the 
nature and importance of statistical interactions 
is indicated. The measurement of statistical 
interactions and concepts affecting t^eir inter- 
pretations are discussed. 

6. Michael Scriven, "Evaluation Skills." 

A discussion of major problems faced by the eval- 
uator and some of the very different answers • 
favored by contemporary evaluation theorists. 

The problem of evaluating this cassette is used 
as an example (amongst others) and the author 
undertakes to refund its cost to the best critic 
each year. 

7. Robert Glaser, "The Psychology of Learning and Instruc- 

tional Psychology." 

Professor Glaser draws major distinctions between 
the phenomena of learning and specific tactics 
designed, to accomplish instructional changes in 
learners. Research implications of these distinc- 
tions are considered. 

8* Fred N. Kerlinger, "Multiple Regression Analysis." 

The audiotape defines and illustrates multiple 
regression analysis, points out its generality, 
power, and ^ide applicability in educational 
research, and emphasizes that it is part\of a 
different way of thinking about research 'and 
/") research problems. The close relation between 
multiple regression and analysis of variance, 
especially multiple regression's ability to do 
all the analysis of variance can do — and more, 
and its ability to handle both experimental and 
nonexperimental data and both continuous and cate- 
gorical variables are discussed. ^ 
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9. Ernst Rothkopf , "Control of Student Activity in Learning." 

This tape contains discussion of the relative 
importance of content, structure and process 

instructional .research. Research • 

on mathemagenic activities (e.g., study activi- 
ties) is also reviewed. 

10. Robert M.W. Travers, "The Lima. tat i ohsof Variables 

'* Derived from Common Language. " 

Discusses some of the problems involved in devel- 
oping a scientific language for describing educa- 
tional phenomena and the difficulties of using 
common language for this purpose. 



The distribution of' the tapes from their initial release until 
November 15, 1971, is presented in Table 1. 



Table- 1. Distribution of AERA Cassette Tapes 
May, 1971 to Mid-November, 1971. 

V ' ' ’ 

Tape Number Numbers Sold 





InNrable 2 
presentedx^hich 

\ 




275 

100 

137 

76 

98 

216 

192 

103 

124 

91 



Total 1,412 

/ 



other information regarding the tape sales is 
may be of interestf'to the reader. 



Table 2. Distribution Information Regarding 
the Cassette Tapes 



Number of cassettes ordered by individuals * 656 

Number of cassettes ordered by institutions 756 

Number of cassettes ordered by members 624 

Number of cassettes orderd by non-members 788 

Domestic orders, number of cassettes 1,245 

Canadian orders, number of cassettes 105 

Other countries, number of cassettes 62 
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Sometime earlier an external evaluation had been commis- 
sioned artd this was to be conducted by Professor Jerry L. Brown 
of Indiana University. The report of that evaluation appears 
as a separate section in the current document. 

The '"'informal reaction gathered by the director regarding the 
tapes was quite positive with one exception, nhmely, the recording 
quality of several of the tapes. Apparently oun: advice secured 
• early in the project from the American Tape Duplicators technical 
advisors was somewhat less accurate than we had hoped. Several 
of the tapes, recorded at local bases, turned out to be of less 
than, satisfactory audio, ^ quality. They were not as loud or as 
clear aS we had hoped. In one case, one of the presenter's 
intermittently used the pause button on the side of the micro- 
phone (an event we had not anticipated) which left a number of 
small bleep sounds in the tape. Most of these were eliminated by 
audio-editing, but the technical quality of that tape and several 
of the other tapes left something to be desired. 

July, 1971 . It will be recalled that the decision to have 
only one instead of two recording sessions in Los Angeles made 
available additional funds. It was decided to add more audio- 
tapes to the series as a consequence f these resources. Accord- 
ingly, consideration was given to methods for securing additional 
presentations at a reasonable cost. Among other alternatives, the 
prospect of a meeting in November of the California Educational 
Research Association and of tying up new presentations with that 
meeting was considered, it was decided to vary the tape prepar- 
ation procedures by inviting participants to make .final presenta- 
tions on November 19th in San Diego, concurrent with the. meeting - 

of the California Educational Research Association, having first 
field tested a preliminary version of their presentation on one 
or more learners. By doing the final recordings in San Diego under 
competent technical supervision, we believed we could alleviate 
the problepi-of inadequate audio quality. At the same time, there 
wereueertain appealing features in having the recorded presenta- 
--tions in hand when the presenters were finished, thereby avoiding 
the difficulties encountered with a few of the initial ten pre- 
senters, where the tapes were secured two months after the agreed 
upon deadline! 

Two other factors entered into the plans for the recording 
of the final four tapes. First, we did not have as much money 
left, so were able to offer only a $250 honorarium to the partici- 
pants along with requisite travel expenses. A second consideration, 
also emerging as a consequence of a relatively tight budget , sug- 
gested that people located somewhat close to Southern California 
be secured, thereby reducing travel expenses. 

As a consequence of these considerations, a re-examination of 
the original list of 50 potential contributors made it possible 
to identify five individuals who had received a respectable number 
of ballots from the AERA Task Force and who were also located in 
California. Comparable letters of invitation were issued to these 
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individuals and four acceptances were secured. The individuals 
participating in the final recording sessions were Dr. Ned A.' 
Flanders of the Far Wes-t/ Regional Educational Laboratory__in Berk- 
eley, Dr. Henry M. Levin of Stanford University, Dr. Reginald L. 
Jones of the University of California at Riverside, and Dr. 

Leonard Marascuilo of the University of California at Berkeley. 

November , 1971 . On Friday evening, November 19, 1971 at the 
California Educational Research Association two concurrent sessions 
were set up whereby one hour each was scheduled for participants. 

The initial half hour was used for the actual recording and the 
final half hour used for discussion with CERA participants who 
attended the session. 

We were attempting to create an atmosphere similar to that 
present for the earlier series, but with a lower budget, in an 
effort to achieve the "communication with an unseen audience" aura, 
two three-panel portable screens were used to shield the presenter 
from the audience; In other words, the person making the presen- 
tation was located at a front table along with recording equipment, 
but was screened from the rest of the audience until the recording 
was completed. At that time the screen was removed and the half 
hour interaction with the audience provided an opportunity for the 
people attending the session to question the presenter. 

All four taped presentations were transmitted to AERA central 
office (for subsequent release along with the initial ten tapes)/ 
within three days after they were originally recorded (safety cqpies 
having first been made to insure against the possibility of loss 
during jnailing ) . a title and brief description of these cassettes 
is presented below: 7 

1. Ned A. Flanders, "Instant Feedback Display^ Based on 

Interaction Analysis." 

t 

This presentation features a discussion on how the 
analysis of one's own teaching behavior provides 
practice in inquiry. The role of this skill in 
conducting research regarding teacher's behavior 
is described. 

/ ■ 

2. Henry M. Levin, "A Research Agenda for the Reform of 

Educational Finance." 

Recent court decisions and pending* litigation 
promise to alter substantially the American sys- 
tem of financing education. This discussion 
attempts to delineate the rasearbh implications 
of these events for improving educational finance. ; 

3. Leonard Marascuilo, "Recent Trends and Developments in 

Fonparamettic Statistics."' / 

, 

Recent trends in the analysis of multidimensional/ 
contingency tables and nonparametric rank analysis 
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of variance are explained and discussed for use 
in the behavioral scliences. Both planned and 
post hoc comparisons! are discussed and evaluated. 

4. Reginald L. Jones, "Research on the Education of the 

Mentally Retarded. " 

A discussion of problems of conducting research 
with educ'able mentally retarded children. Dis- 
tinctive. problems encountered when carrying out 
experimental studies with such learners are 
examined. ^ 



CONCLUSION 



In review, seven hours of instructional materials designed 
for the continuing education of educatiortal . researchers were 
produced as a consequence of this project. The cost of these 
materials was somewhat less than $2,000 per hour of instruction. 

In some ways, this might be considered relatively expensive 
material development.. On the other hand, one of the reasons we 
chose to select high stature figures in the field of educational 
-research was that their very participation in the training opera- 
tion would be more likely to attract listeners . 

To put it in another way, if an educational researcher (in 
need of upgrading) were given the opportunity jto listen ' to a 
half hour instructional tape prepared by someone he had never 
heard of , and a half hour instructional tape prepared by someone 
of Michael Scriven's stature, we felt that many individuals might 
prefer to hear Scriven's personal uttered views. The chief devel- 
opmental costs, therefore, were for honoraria and travel expenses 
for these high stature educational researchers. 

As an instance of low cost development, the present project 
seems to have accomplished its mission. 

Concerning the evaluation of the enterprise, there are two 
considerations. First, it might have turned out that these tapes, 
once they had been made available to the educational research ' 
committee, would not have been used. This kind of response would 
certainly have settled the question of the utility of cassette 
recordings with some certainty. If people don't buy the tapes, 
then obviously they aren't going to be of much use. With the 
present cassette tapes, however, it appears that this is not the 
case. During the almost eight month period of their availability, 
over 1,400 tapes were sold. There is no reason to believe that J 
these sales will decline dramatically in the near future, given 
reasonable promotional efforts on the part of^ERA. 

/ 
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The second question regarding the quality of the cassette 
tape enterprise must be based , however, on an evaluation of the 
degree to which the tapes, once used, accomplished their mission. 
This was the concern of Professor Brown, an external evaluator, 
and will be dealt with in the next section of this final report. 
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Introduction 



/ / 

/ As the number of researchers engaging in educational 

research grows and as the ra^e of discovery of new knowledge 
increases, providing the educational community with updated 
knowledge and skills becomes\increasingly difficult. One 
method which has been suggested^ as being potentially useful 
in disseminating knowledge and skills is using prerecorded 
audio— tape cassettes as a means of instruction. Prerecorded 
cassettes have much appeal. They are not prohibitively ex- 
pensive for individuals to purchase, are easily used in a 
variety of situations, and are easily stored. Moreover, unlike 
text materials, they can be used in situations (such as when 
driving) when the use of visually-oriented instructional 
materials is not feasible. 

To determine if a prerecorded library of cassette-tapes 
would be useful to educational researchers, ten outstanding 
researchers were invited to contribute to a cassette-tape 
library to be produced by AERA. The production of the ten 
tapes for this library was directed by w. James Popham (School 
of Education, UCLA) and financed by a grant frpm the Office 
of Education. The 'tapes were recorded in the spring of 1971. 
They are now available from AERA and may be purchased indi- 
vidually or in combination for $6.00 per tape, with a small 
discount being given purchasers who order complete sets. 



Evaluation 

To determine user-perception of the effectiveness of the 
cassette-tape library in promoting new knowledge and skills, 
provision was made in AERA's proposal to the Office of Edu- 
cation for outside evaluation of the' project. The proposal 
called for a two-stage evaluation. The first stage was to 
occur approximately a month after, distribution of the tapes 
began. The goal of this portion of the evaluation was to * 
find out if users thought the tapes valuable. The purpose 
of the second stage was to ascertain if purchasers thought 
the tapes made any difference in the way they went about 
their professional work.* The information-gathering activities 
related to this stage were to occur about six months after 
distribution of the tapes began. Given the original produc- 
tion schedule for the tapes, the first stage of the evaluation 
would have occurred in , May, 1971, and the second stage would 
have occurred in October. Because of production delays it 
was not possible to follow this schedule and, consequently, 
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the evaluation outlined in AERA's proposal to the Office of 
Education was modified.* 

\ 

The modified design had the following features. Included 
with each order of tapes shipped by AERA was a postcard ques- 
tionnaire designed primarily to identify individual tape pur- 
chasers by name,** although it also inquired into how they 
learned about the cassette library and how they planned to 
use it. Then , in September, a formal (anonymous) questionnaire 
was mailed to each of the 211 individuals who, on the basis 
°f information provided by AERA and postcard questionnaires 
which had been returned, could be identified as tape purchasers.*** 



Evaluation Goals 

^H-hough the plan by which the cassette tape library 
was to be evaluated changed, the primary goal of the evalua- 
tion study was that or igina lly proposed by AERA to the Office 



* If carried ou£ as planned, the first stage of the eval- 
uation would have occurred when many purchasers were vacationing 
arid the second stage would have occurred after the project's 
life had expired. Given that the project's contract had 
been previously extended, the project was of relatively modest 
proportions, and that modification' of the proposed evaluation 
design .would produce few costs .compared to the costs involved 
in securing a contract extension, a different evaluation stra- 
tegy was suggested by -the evaluator. 

** Because individuals often order tapes through a pur- 
chasing department or library, especially when grant funds 
are involved, it is often difficult to identify who is ordering 
the tapes. The postcard questionnaires attempted to circum- 
vent this’ difficulty, albeit unsuccessfully since many pur- 
chasers did not return the postcard. 

A 

. The AERA list contained the names and addresses of 
295 individuals and institutions .who purchased tapes in the 
period between March 25th and July 19, 1971. According to 
information supplied by AERA 1,038 tapes were sold to these 
purchasers. The 211 individuals who were sent questionnaires 
accounted for approximately 632 tapes. As with the formal 
. questionnaire, the response rate for the postcard questionnaire 
v W3S poor. As of September, 35 postcards ha& been returned. ~ 

If July's figure of 295 orders is used as /the base, this 
yields a response rate of 12 percent for the. post card ques- 
tionnaire. Since more than 295 orders had been received by 
September, the actual response rate is even lower. 
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of Ed uca ti° n: To determine if tape purchasers found the cas 

settes useful. In addition to this primary goal , the eval- 
uator suggested that a number of secondary questions be ad- 
dressed. These questions pertained to .the nature of the 
group purchasing the cassettes, the audience reached, sug- 
gestions for improvement of the cassettes, and suppon for 
cassette-based instruction relative to alternative methods 
o in-service education. Specif ic " questions which the eval- 
uation study addressed included these: •; 

a. Will the cassette-tapes be purchased? 

b. Who purchased the cassettes? 



c. Who listened to the cassettes? 

d. Are the cassettes perceived as being effective in- 
structional devices? 



e. 



f. 



Is the quality of production perceived as beina 
adequate? . ' 

r 

Would additional tapes ,be .purchased if available? 

As can be seen from the. /list above, determination of the 
actual instructional effectiveness of the tapes is not an 
evaluation goal. Except for question "a" in the list above 

3 ^ Udy do 9 s not attempt to go beyond user per- 
ception of effectiveness. Nor does it attempt to go beyond 
user-perception of quality and adequacy of content or means 
of production. Readers who are interested in judgments in 
hese areas or who are interested in considering the trade- 
offs involved in. deciding to produce or purchase cassette- 
tapes for inservice training are referred to a paper by Gene 
V. Giass, A Product Evaluation of 'Evaluation Skills 'Cas- 
sette Recording 6B (By Michael Scriven) - An Instructional 
tion ?£° duced by the American Educational Research Associa- 



Data Limitations 

An attempt was made to identify and elicit responses ' 



* Thanks are due Dr. Glass for allowing me to see his 
paper while in draft. Readers wishing a copy of Dr. Glass' 
paper might best fare by writing Dr. Glass at the Laboratory 
of Educational Research, University of Colorado. The date 
on Dr. Glass' paper is July, 1971. 
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^h° l ? ad ord ered tapes between March and 
Kk-tT^ed ? Ve " rndividuals were successfully 

-?s%f^rti r kc h T ?^ sa s„“£ ' 

f° r the Kerl? "9e? tape forS' 
wtf.on -,f ' J 4 ‘? % of aales - Because only a small pro- 
Sndid t^ £ th« P purchasers are represented by those who re s- 
^ ^ uestlo faire and because responsiveness is 

*'0 ' ) co^v to l^ n Z r ^ nd0 t factor6 < the results of the survey 
.^ely to be subject to considerable error.* 



Demand 



ms can be- seen in Table 1, by September, when the ques- 
t onnai res were distributed, AERA had sold 1,349 tapes. q 
Six hundred thirty-six (47%) of these tapes were sold directlv 
to .naividuais, the remaining 736 (53%) were either sold > * 

d-rec^v^n ? 1 " dlvlduals through institutional orders or 
Usi^of to institutions themselves.*** Six hundred thirteen 

WGre P urchased by AERA members. Total 

to S9 676 00 he nf d ^ S ' if ° f Se P teraber 1 > 1971, amounted 
t^h '°°k 0f . fc £ G 10 ta P es Produced for the library, 

As af b ^tTL° n behavioral objectives was the most popular. 
?oyt .n ^ r \ U accounted for 18 percent of total sales. 
^ xt ^Popularity was the tape on evaluation by Scriven 

sniH aoer \ ln Tabie 1* 204 of these tapes (15%) had been 

sold by September. The third most popular tape as of SeD- 

r . tha ^ Glaser dealing with instructional psycho- 

- g> . Q..e nundred eighty-three of these tapes had been sold 

KLl!naS^i r ^nsdA? e3 0 M-i? SSer P°P ula £#1^ included those by 
Acrnnge. , Luiasdaine, Hillmjui and Tr^Cfers. y 



__ „„ Because of financial and temporal restraints and be- 
cause of a desire to protect the anonymity of respondents 
and nonrespondents, no attempt was made to determine the 

ri«^i n 2 U1 f hln9 character istics of the respondents and non- 
toapondents. nor was any attempt at a follow-up mailing made. 

uoon fhoTt 109 e i figUrea A Sre su PP lied b y AERE and are based 
upon their sales records. 
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Of the 1,349 tapes sold by September, 636 (47%) had been 
sold to individuals, 611 (96%) of whom were members of AERA. 
Domestic orders accounted for 1,190 (88%) of all cassettes 
sold. Canadian orders accounted for 98 (7%) of all cassettes 
sold, with other foreign . countries accounting for the 'remaining 
61 passettes (5% of the total number sold). 



I 



Table 1. Sales by Author 
As Of September, 1971. 



Author 



No. Sold' 



% of Total Sales 



Gagn^ 


262 • 


19 


Glaser 


183 


14 


Kerlinger 


100 

\ 


7 


Lumsdaine 


\ 72 


5 


Meyer 


96 


7 


Mi liman 


94 


7 


Platt 


132 


9 


Rothkopf 


120 


9 


Scriven 


204 


15 


Travers 


86 


_6 










1,349 


98* 


Note — Because 
100%. 


of rounding percentages do 
Source : AERA 


not sum to 



Purchaser Characteristics 

Of the 49 respondents who returned the questionnaire, 

47 percent indicated that -they would primarily use the tapes 
in a college or university setting. Thirty-one percent indi- 
cated they would use the tapes within a school district. 
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Ten percent indicated that they would use the tapes in a 
regional lab or R & D center, and another ten percent indi- 
cated that they would primarily use the tapes in a private 
educational organization. The remaining two percent of the 
respondents indicated that the tapes would be used in a state 
department of education. 

Table 2 relates to respondents professional interests. 

As can be seen in the table, research, development, evaluation 
srid instruction are most frequently cited as being areas of 
primary interest. Diffusion/dissemination activities and 
management are infrequently cited areas of interest. 



Table 4 2. Respondents 
Primary Prbfessional Interests. 



What is your primary area of interest? 

Research _41_% 

Development 27 

Dissemination ip 

Evaluation 35 

Instruction 33 

Management 14 

Other q ' 

No Response p 

168 % 

(N = 49) 

* / 

Asked how they' preferred to receive information, 27 
percent of the respondents indicated they preferred audio 
tapes over the alternatives presented them. This was rela- 
tively the most popular way of receiving information. Next 
in popularity was journal articles and abstracts. Table 
3 presents complete data in regard to this question. 
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Table 3. Preferred Media. 




Which N medium do you prefer for receiving 
information? 

Convention paper sessions 



or workshops 22 % 

Video tapes 10 

Films 6 

Audio tapes 55 

Books and manuals < ■ 20 

Journal articles and 

abstracts 39 

No Response 8 

160 % 
(N = 49) 



Seventy-one percent of the respondents indicated that 
they owned their own cassette player. Twenty-seven percent 
did not (with two percent not responding). Data on where 
respondents typically listen to the tapes is presented in 
Table 4. As can be seen there, the office and the home are 
the most popular locations.* Relatively few respondents 
listen to tapes while commuting, while in a media center, 
or while in a classroom. 



* The popularity of cassettes and journal articles as 
means of receiving information may be related to their abil- 
ity to be used at the office and at home. 



Table 4. Setting In Which Respondent 
Typically Listens to Cassette-Tapes. 



Where do you usually listen to the cassettes? 



At home 39 % 

At the office 55 

In a classroom 6 

In a library or media center • 14 
In a car, plane, etc. 14 

128 % 



(N = 49) 

*Note — Percentages sum to more than 100% 
because of multiple-responding 



Audience 



Purchasers who responded to the questionnaire accounted 
for 172 tapes. That tapes were purchased, however, does not 
mean that they were listened to. Thus the audience for the 
tapes may be smaller than that which would be inferred on 
the basis of sales. On the other hand, the tapes could reach 
a larger audience them would be indicated by sales data if 
they were played for others besides the purchaser or if the 
purchaser had them duplicated. * 

Of the 172 tapes purchased by those who responded to 
the questionnaire, respondents had listened to 105 (61 per- 
cent). The breakdown by author is given in Table 5. There 
it can be seen that these tapes were listened to by\ over 
70 percent of their purchasers: Platt, Glaser and Kerlin- 

ger.^ Sixty-five percent of the respondents who bought the 
Gagne tape on behavioral objectives and learning hierarchies, 
the best selling tape, listened to it. Among the tapes least 



* Since production of the AERA tapes was financed by the 
Office of Education, the tapes are in the public domain and 
may be legally duplicated. 
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listened to by their purchasers were those by Lumsdaine and 
Travers.* Only about a third of the respondents who had pur- 
chased these tapes listened to them. 



Table 5. Number of Tapes Pv Author Ordered 
By Respondents and Percent of Tapes By Author 
i Ordered and To Which Respondents Listened 



Tape 


No 


i. Ordered 


% 


(Bv Author) 


by_ 


Respondents 


Listened To 


s 








Gagne 




31 


65% 


Glaser 




23 


74 


Kerlinger 




22 


73 


* Lumsdaine 




9 


33 


Meyer 




13 


54 


Mi liman 




13 


62 ' 


Platt 




13 


77 


Rothkopf 




11 

< % 


55 


Scriven 




24 


54 


Travers 




13 


39 


Total 




172 


61% 


* Note — The 
the others. 


Lumsdaine 


tape was released 


later than 



Seven (4%) of the AERA tapes were duplicated (or were going 
to be duplicated) by the respondents. Twenty (12%) of the 
tapes" were played for students of the respondents. , The break- 
down on these by tape is( presented in Table 6. /' 
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Table 6. Percent of Tapes By Author 
Purchased By Respondents and Duplicated 
Or Used in Instructional Situations. 



Author 


% Duplicated 


% Used in 

Instructional 

Situations 


N 


Gagne” 


9% 


/ 16% 


31 


Glaser 


9 


17 


23 


Kerlinger 


- 


14 


22 


Lumsdaine 


- 


11 


9 


Meyer 


- 


8 


13 


Mi liman 


- 


15 


13 


Platt . 


8 


- 


13 


Rothkopf 


- 


18 


11 


Scriven 


4 


4 


24 


Travers 


. . 


8 


13 


Total 


4% 


12% 


172 



Effectiveness 

The purpose for developing the tape library was to provide 
professional educators with an interesting and effective means 
of improving their professional skills. The questionnaire 
contained a number of items which attempted to assess the 
respondents attitude regarding the ability of the tapes to 
reach this goal. Respondents were asked to consider the 
interest-maintaining quality of the presentation, the adequacy 
of the content covered, the instructional effectiveness of 
the presentation, and the presentation's ability to stimulate 
further inquiry. Respondents who used the tapes in instruc- 
tional situations were also asked to evaluate the tapes 
effectiveness in regard to perceived impact on their students. 
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Table 7 presents data on responses to positive statements 
about each tape. In Table 8 the reaction of respondents to 
negative statements about the tapes is shown. As can be 
seen by comparing the tables, respondents were more positive . 
about the tapes than negative. All but four tapes were con- 
sidered as having interesting presentations by 50% or more 
of their listeners. The exceptions were the tapes by Glaser, 
Lumsdaine, Platt and Scriven. Six tapes, those by Kerlin- 
I ger, Lumsdaine, Meyer, Millman, Rothkopf and Scriven, were 

reported to have increased understanding of the topic cov- 
1 ered by 50% or more of their listeners. One tape was reported 

‘ as having motivated further study by 50% or more of its lis- 

teners. This was the tape by Lumsdaine. Approximately a 
' third of the listeners for the Gagn£, Glaser and Kerlinger 

tapes reported that further study of the topic was motivated 
by the tape. 



Table 7. Agreement With Positive 
Statements About the Tapes by Respondents 
Who Listened to Those Tapes. 



Author 


Presentation 

Interestinq 


Content and 
Coverage 
Adequate 


Increased 

Understandinq 


Motivated 

Further 

Study 


N 


Gagn^ 


60% 


85% 


45% 


40% 


20 


Glaser 


35 


71 


47 


35 


17 


Kerlinger 


69 


63 


50 


38 


16 


Lumsdaine 


33 


67 


67 


67 


3 


Meyer 


57 


57 


71 


14 


7 


Millman 


63 


25 


50 


13 


8 


Platt 


30 


40 


20 


20 


10 


Rothkopf 


67 


67 


67 


17 


6 


Scriven 


62 


46 


54 


8 


13 


Travers 


20 


60 


20 


_ 


5 
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Table 8. Agreement With Negative 
Statements about the Tapes by Respondents 
• Who Listened to Those Tapes 



Author 


Not 

Informative 


Lacked 

Clarity 


Would Not 
Recommend Tape 
Be Purchased 


N 


Gagne* 


-% 


5% 


15% 


20 


Glaser 


- 12 


- 


18 


17 


Kerlinger 


- 


- 


* 6 


16 


Lumsdaine 


- 


- 


33 


3 


Meyer 


- 


- 


14 


7 


Millman 


13 


- 


13 


8 


Platt 


50 


10 


50 


10 


Rothkopf 


- 


- 


17 


6 


Scriven 


15 


8 


23 


13 


Travers 


20 


- 


40 


5 



As to tapes which would not be recommended for purchase 
by their listeners , three, those by Lumsdaine ( Platt and Tra- 
vers, were the only tapes to have a third or more of their 
listeners indicate that they would not recommend purchasing 
the tape. About 20 percent of those who listened to the 
Scriven tape indicated that they would not recommend that the 
tape be purchased. For the Gagn^, Glaser and Rothkopf tapes, 
between 15-19% of their listeners indicated they would not 
recommend the tape be purchased. Between 9-14% of those who 
listened to the Meyer and Millman tapes indicated they would 
not recommend them. Six percent of those who listened to 
the Kerlinger tape indicated that they would not recommend 
its purchase. 



Respondents who used the tapes in instructional situations 
were asked to subjectively evaluate the effect the tapes 
had on their students. They indicated that of the 20 tapes 
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played, 18 (90%) seemed to effectively increase student un- 
derstanding. Seventeen (85%) provoked student initiated 
discussion, and nine (45%) stimulated further independent 
,or group study of the topic discussed. Respondents indicated 
that they planned to incorporate each of the 20 tapes played 
into their regular instruction, although evaluation instru- 
ments had been developed by the respondents for only four 
of the 20 tapes. The breakdown of these responses by tape 
is presented in Table 9. 



Table 9. Agreement With 
Statements Regarding Each Tape's 
Effectiveness in Instructional Settings. 



Author 


Increased 

Student 

Understandinq 


Provoked 

Discussion 


Stimulated 

Further 

Study 


Plan To 
Use 

Reqularlv 


N 


Gagne 


100% 


100% 


80% 


100% 


5 


Glaser 


100 


100 


50 


100 


4 


Kerlinger 


67 


67 


- 


67 


3 


Lumsdaine 


100 


100 


- 


100 


1 


Meyer 


100 


- 


•; - 


100 


1 


Mi liman 


50 


100 


50 


50 


2 


Platt 




- 


- 


- 


0 


Rothkopf 


100 


50 


50 


100 


2 


Scriven 


100 


100 


- 


- 


1 


Travers 


100 


100 


100 


100 





Format Changes 

Audio-tapes are one possible means of communicating in- 
formation to professional educators. Depending upon the 
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audience being targeted and the information attempting to be 
transmitted they may be appropriate or inappropriate to the 
task, sufficient or insufficient. Tape listeners were asked 
to eyaluate the tapes in regard to these factors: (a) Would 

a printed format have been more useful than a taped format? 
(b) Should supplementary materials such as a workbook be 
included with the tapes? (c ) Should a transcript have accom 
panied the tapes? Responses to these questions are summar- 
ized in Table 10. " 



Table 10. Percent of Respondents Who 
Listened to Tapes and Who 
Agreed with Suggested Format Changes. 



Printed Format 



Include 

Supplementary Include 

. Material Transcript 



14% 

(n = 105) 



9 % 

(n = 105) 



28% 

(n = 105) 



As can be seen in the table, 14% of the respondents who 
listened to the tapes indicated that a printed format would 
have been more useful. Nine percent of the respondents thought 
supplementary materials should have been included.* Twenty- 
eight percent thought a transcript should have been included. 



Production Control 

Purchasers who listened to the tapes were asked to rate 
the quality of productiort in regard to the need for profes- 
sional narration, need for editing, and overall technical 
quality. As can be seen from the data in Table 11, technical 
problems were not an issue for most listeners, with only 
six percent indicating it was needed. A third of those lis- 
tening to the Scriven tape, however, did feel that the tape, 
which was interrupted by a series of " s top /st arts " , had tech- 
nical qualities wh ich de tracted from the subs tance of the 



P l^ C !? a v e ^ w ~° had listened to the Mi liman tape com- 
prised about half of the persons who wanted supplementary 
materials Listeners to the GagnS, Ketlinger and^lyer^apes 
accounted for the remaining few tapes. y P 




- 32 - 




tk 



presentation. Professional narration and editing was indi- 
cated as being required in only a few cases , the most notable 
being the Travers tape (20% of its listeners indicated it 
needed professional narration and 40% indicated it needed 
professional editing).* 



Table 11. Evaluation of Technical 
Aspects of Tapes by Respondents 
Who Had Listened to the Tapes. 



Technical Quality 
Detracted From 
Author Substance 


Professional 

Narration 

Needed 


Editing 

Needed 


N 


_ / 
Gagne- 


10% 


10% 


15% 


20 


Glaser 


12 


6 • 


12 


17 


Kerlinger 


6 


- 


6 


16 


Lumsdaine 


- 


- 


- 


3 


Meyer 


14 


14 


/ 14 


7 


Millman 


13 


13 / 


13 


8 


Platt 


0 


/■ * 

10 


0 


10 


Rothkopf 


0 


0 


- 


6 


Scriven 


31 


- 


8 


13 


Travers 


20 


20 


40 


5 



Topic Preference 

Tape purchasers were asked to select from a group of 
topics those they would most like to see presented on audio 



* Six percent of the listeners agreed that professional 
narration was needed and 10% agreed that professional editing 
was needed. 
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tapes. Response data is presented in Table 12. As can be 
seen in the table the summaries of current research findings 
is the topic most respondents would prefer to have presented 
on tape. Close to this in popularity is discussions between 
prominent advocates with different points of view. "How- 
to-do-its" and discussions of "hot" issues are next in popu- 
larity. Discussions of recent funding, laws and politices 
is the least popular of the alternatives. 



* 

Table 12. Preferred Topics. 



Which of the topics would you most like to see 
presented on audio tapes? 

"How-to-do-its" (demonstrations of 37 % 

technique ) 



Dialogues between prominent advocates 
with different points of view 49 

Discussion of current "hot" issties 37 



Summaries of current research 
findings 

Discussions of recent findings, law, 
and policy 

No Response 



Sales Potential 



55 



20 

4 

198 % 

(N = 49) 



Related to the discussion above, tape purchasers were 
asked if they would be willing to purchase audio- tapes again 
Approximately 89% of the respondents answered "Yes" , they 
would (the remainder said "No", with no nonrespondents). 

As to whether they'd be willing to pay $30-$45 for a six 
month trial subscription for a cassette-based "journal" of 
education, about 35 percent indicated that they would be 
willing. Another 37^ percent indicated that they were unsure 
and .the remaining 29 percent said they wouldn't. 
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Data Summary 

r 

"As of September, 1971, AERA sold 1,349 tapes. Forty- 
seven percent of these were sold directly to individuals. 

The rest were sold either indirectly to individuals or directly 
to institutions. Of the ten tapes, that by Gagne^ was the 
most popular, having sold 204 copies (18% of sales) by Sep- 
tember. Tapes by - Scriven and Glaser v.'ore next in popularity. 

To determine user reaction to the tapes , an anonymous 
questionnaire was mailed to 211 individuals who -could be 
identified as tape purchasers as of September. Usable ques- 
tionnaires were returned by 49 individuals. These individ- 
uals accounted for 13% of the tapes which had been sold at 
that time. With the exception of the Kerlinger tape, of whose 
sales respondents accounted for about 22%, respondents accounted 
for 10-15% of sales for each tape in the series. 

Respondents indicated they would use the tapes primarily 
in a college or university setting. Most indicated that re- 
search was their area of primary interest with evaluation 
and instruction also being areas of primary concern. Pre- 
sented with alternative ways of receiving new information 
and skills more respondents preferred audio-tapes than any 
other means. Next in preference were journal articles 'and 
abstracts. The office and home were typically used for- lis- 
tening to the tapes , with relatively few- respondents listening 
to the tapes while commuting or while in a "public" setting 
such as .a media center or classroom. 

Purchasers tended to listen to the tapes they bought, 
listening to 61% of the tapes purchased, Only two tapes, 
that by Lujnsdaine (which was released later than the other 
tapes ) and that by Travers, had less than 50% of their pur- 
chasers listen to them. Respondents who listened to the 
tapes responded in greater proportion to positive statements 
about the tapes .than to negative statements about the tapes. 

In general respondents indicated that the tapes increased 
their understanding of the topic discussed, although rela- 
tively few were stimulated to further inquiry. Professional ' 
narration or editirig was indicated as being needed by only 
a small percentage of the tape listeners . • Only for the Scriven 
tape did a sizable proportion of listeners (about 20%) feel 
that technical quality detracted from the substance of the 
presentation. Format changes, such as using a printed format 
for conveying information rather than audio-cassettes, were 
supported by only a small percentage of the listeners. About 
® however, did indicate that the inclusion of a trans- 

cript would be useful. • 



/ 



35 



As to topics respondents wanted to have presented on tape, 
summaries of current research findings, debates and discussions 
of cu: rent , hot 1 issues were the three 'choices most' f avored 
•ivy respondents. 



Ev alualor ‘ s C o rumen t a r v •' 

/ # > 

The data presented above are the result of a rather un- 
soph is t icatod effort to ascertain user-reaction to the cas- 
settes in the AERA Series B library. This library was de- 
veloped to test the viability of using cassette-tapes to 
transmit knowledge and skills to educators primarily engaged 
in educational research. If sales are used as » the criterion 
for success . then the project must be judged as relatively ° • 

successful. Similarly if response data from purchasers replying 
to the questionnaire is used, the project succeeded in meeting 
its goals. • Purchasers were primarily interested °in research , ' 
so the proper target audience was reached’. Moreover, they 
found that the tapes increased their understanding of the 
topics presented* and, in a few cases, motivated "furthe’r study. 

Unfortunately because the tape purchasers were reluctarit 0 
to complete and return the questionnaire that, was mailed 
them, it is difficult to determine whether or net the sales' 
data or the respondents' responses are valid indicators of 
AERA “ b success in meeting its goals. The issue • is further „ 

coiiipl icated since no attempt was made to determine the actual-? 

inst ructiona^j effect of the ten tapes in the series. Finan- 
cial restraints. . among other things, prevented this. Simi- 
larly technical aspects pertaining to the content presented 
and the quality and means of presentation werd not evaluated. 

* ' % a * . 

< it nevertheless appears that audio-tape cassettes are 

viable way of making researchers aware of new trends in 
educational research and evaluation. It would seem, however, 
that if more than awareness were desired, then changes in 
format would be necessary. Take, for example, the tape by 
Gagne. This tape is probably the most skill-oriented of the 
tapes in th# library.. Its objective appears to be to <teach- 
the listener to write behavioral objectives that are indi- 
cative of the hierarchical level to which they relate. This 
is a complex task that requires the learner to be able to 
write* behavioral objectives, to be able to identify the levels 
m a Gagne hierarchy and associate each level with a code- I. ' 
word provided by Gagne, and to write a behavioral objective 
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that indicates, through use of the code word, the level of 
the hierarchy to which it relates. A complex task such as \ 
typically can't be achieved through listening. Unless 
the, learner possesses a high degree of previous knowledge 
about each, of the subordinate tasks, he must do more than 
learn the code words. He must learn how to do all the other 
tasks a S; well (and the subordinates that they entail). For 
this, the learner needs practice both on individual compo- 
nents taken separately and in combination. Practice such 
as this necessitates that the tape be restructured into a 
semi-programmed format _or__that-an adjunct program, such as 
a wookbook be' included. * 

Since, however, most respondents indicated that they-pre— 
tapes oriented toward "awareness" ," a workbook or 'active 
response program is not required. Simply listening to the / 
tape or reading a transcript is sufficient. That only ten 7 
P® r P®r*’t of the respondents who listened to the Gagne tape / 
felt the need for a workbook, suggests that most either had / 
a high level of previous knowledge or were not interested / 
in moving .beyond the awareness stage. That so few of the / 
listeners sought out other references as a result of listening 
the tape, suggests that they were oriented towards aware- 
ness rather than skills. This might explain more respondents 
wanting (additional) tapes summarizing current research findings 
than wanting tapes dealing with tec nique. j 

■ If awareness is all that is required, then a different 
marketing strategy might be employed to increase the size 
of the audience being reached. Tapes are an expensive means 
of promoting awareness compared to printed materials. Yet 
they ("were favored, by questionnaire respondents. Offering 
an ootion to those\desirous of becoming informed about new 

seems sensible." Those who wish less expensive materials 
or who don't enjoy receiving information via cassette should 
also |be given the opportunity to come into contact with the 
thinking of outstanding men in the field, something not pos- 
sible- the way the Series B library was offered fo£ sale. 

This could be done by offering researchers the opportunity 
to purchase typescript versions of the presentations contained 
on the cassettes. Since almost 30 percent of /the respondents 
indicated that a transcript should hav e accompanied the tapes, 

” 

* This assumes that a tape is to remain the primary 
vehicle for communicating with the learner.; 
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it would also allow tape purchasers who desire a transcript 
to have the opportunity of buying both taped and printed 
versions of the same material. This would probably increase 
the number of people reached by the library since purchasers 
would be able to easily and inexpensively duplicate the 
typescripts (as by Xerox) and distribute them to colleagues 
and students. It would therefore seem desirable for AERA 
to offer the library in both cassette and print and, to 
encourage the widest possible 0 distribution, to provide a 
small discount to those who purchase the library in both 
forms. 



Conclusion 

Using audio-tape cassettes to make educational researchers 
aware of new trends in educational research seems to be a 
viable concept. Tcypes were purchased and found useful by 
th®i^ users.- Most indicated that they would purchase tapes 
again. There is some question, however, of the effective- 
ness of using cassettes without supporting materials if non- 
awareness objectives, especially those relating to skill 
performance, are to be attained Assuming awareness is 
the primary goal for purchasers, then providing the purchaser 
with the opportunity to buy inexpensive printed versions 
of the presentations contained in the library (either as 
a substitute for the cassette or in addition to it)would 
seem to be an effective means for reaching the largest pos- 
sible audience . 
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December 10, 1971 



Dr. Michael Scriven 
233 Moses Hall 
University of California 
Berkeley, California 

Dear Michael: 



During the past summer the American Educational Research Associa- 
tion received a U.S.O.E. grant to assemble a library of half 
hour cassette tapes dealing with recent advances in the strat- 
egies and tactics of educational research. Ten prominent indiv- 
iduals were to be identified who could prepare a recording which 
would improve the skills of practicing educational researchers. 

AERA is anxious to test the viability of this approach to pro- 
yiding continuting education for the educational research commun- 
ity and will distribute the tapes after they are prepared. 

As director of the project I have been in frequent' consultation 
with the AERA Task Force on the Training of Educational Researchers 
whose members agreed to aid- in the selection of the initial ten 
contributors to the tape library. After several rounds of sugges- 
tions, straw ballots, and actual voting, I hope you will consider 
it a deserved indication of your colleagues 1 support that your 
name was selected. 



X 

y 




I hope you will be- willing- to participate in this project and in 
order for you to make a decision will briefly outline specifics 
of a contributor's involvement. 

You would prepare a 30 minute presentation designed primarily for 
the practicing educational researchers (although such tapes would 
undoubtedly aid prospective researchers as well), the purpose of 
which would be to provide or upgrade one or more research- related 
competencies of the listener. In addition, you would identify 
several references for - follow-up reading, indicating the role of 
each with a one or two sentence annotation. At a three day con- 
ference late this year or early next year Los Angeles, along 
with the other nine contributes , you would, record your remarks 
at the same time they are being heard by an unseen audience of 



(cont ' d) 
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Dr. Scriven 



Page 2 



December 10, 1971 



50 pr so educational researchers in a different room. You would 
only need to attend the conference for one half day, although 
you would be welcome to stay for the whole session. After your 
presentation, you would go to the other room where those who had 
listened to your remarks would (1) offer suggestions for research 
and (2) discuss the substantive aspects of the presentation (which 
might also yield ideas for modifications). .You would be given 
a loaner cassette recorder, return to your home with your original 
recording and within several weeks would record a revised presen- 
tation at your home (office, etc. )'. This would be the tape to 
be included in the AERA library. 

For this contribution you will receive a $500 honorarium and all 
expenses for the conference. The tape will become the property 
of AERA, under whatever arrangements regarding copyright are in 
effect by. U.S.O.E. at that time. 

We believe that if the tapes are well received by the educational 
research community, the cassette library approach will be a use- 
ful way to improve our colleagues' skills. Obviously, I hope you 
will wish to be a charter contributor to the series. 

/ ’ 

Will you please write me as soon as possible indicating whether 
you will participate in the project. We'll need to get an alter- 
nate if you decline. If you accept this invitation, please list 
a phone number so that by mail or telephone X can reach you to 
discuss a topic and date when you might be free to make your 
first recording. 



Sincerely , 



W. James Popham 



WJP/bg 
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APPENDIX 



Postcard Questionnaire 



Re: AERA Cassette Tape Series - "Perspectives on Recent Research.” 

Thank you for your purchase of tapes from the series. To help us better evaluate this 
new service, we would appreciate your completing the card an! returning it to AERA. (R 
postage is necessary) . 

Name Position 

Place of employment • 



AERA Member Yes No 





/ 


CHECK TAPES 


ALREADY ORDERED 






IB Gagne 




4b 


Lumadaine 


7B 


Glaser 


2B Meyer 




5B 


Millman 


8B 


Kerlinger 


3B Platt 




6b 


Sc riven > 


9B 


Rothkopf 








~ 


10B 


Travers 


Reason for purchase? 
Personal use . 


□ 


Class instruction D Library Q Other O 




Information source 
Colleague(s) 


□ 


AERA Flyer D 


Educational Researcher 


□ 


Other □ 



A 



\ 
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ameRiCcin eoucanonaL 
ReseaRCH associanon 



Thank you for recently purchasing some of the cassettes from the 



AERA is interested in improving the quality of its cassette-tape 
library to make it more useful to you, and others like yourself in 
the field. The purpose of this questionnaire, which we hope you'll 
be kind enough to complete, is to seek your reaction to the 
cassettes and to solicit your recommendations for improving the 
cassette-tape library. ! 

Please complete the questionnaire and then return it in the envelope 
provided. No postage is required. Although the questionnaires are 
coded this is only for bookkeeping purposes. As you might expect, 
emonymity of responses is guaranteed. 



AERA Audio-Tape Series, New Trends in Educational Research. 



% 
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Directions 



In the left hand column is a series of questions, statements or phrases 
pertaining to the cassette tapes in AERA Series B. Each tape in the series, 
identified by author, is listed in one of the columns at the right. If a 
statement applies to one or more of the cassettes indicate to which cassette 
it applies by placing a check ark (V) in the column associated with that 
cassette. Answer for as many cassettes as you wish, but do not answer for 
those cassettes which you have not purchased or monitored. TAPES 



O 

ERJC 



1. 

2 . 




Which tapes did you purchase? 



Why did you purchase each of the titles 
you did? 



a) Speaker's reputation 

b) Interest in topic 



c) Other 



(what ) 



Which of the tapes have you listened to since 
receiving your order? 



U. Which of the tapes have you duplicated (or 
plan to duplicate)? 



Which of the tapes do you plan to use in a 
classroom for instructional purposes? .... 



Part II 

If you haven't listened to any of the tapes 
please go to Part [II. For those tapes to 
which you have listened, please decide 
whether or not the statement in the left 
column applies. If it does, place a check 
mark (>/ ) in the appropriate box. 



6. Professional narration needed 



7. Poor technical quality detracted from substance 

8. Interesting presentation 

t 

9 - Not very informative • 
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22. Vfhich tapes seemed to effectively increase 
-your students' understanding of the topics 

/ presented? 

23. Which tapes provoked student initiated 

discussion? 

2b. Which tapes stimulated further independent or 
group study of the topic presented? 

25 • Which of the tapes do you plan to incorporate 
as a regular feature of your instruction? 

26. For which tapes did you develop an evaluation 
instrument? 




.(continued) 



Part IV 



Please answer the questions belcflir: 

27. Where will the tapes primarily be used?. 
_a) In a college or university. 



_b) In a regional lab or R&D center. 

c ) In a State Department of Education. 

d ) In a school district . 

_e) In a private education-oriented organization. 



28. How will the tapes typically be used? 

_a) For the purchaser's personal instruction. 



_b) For use- in pre- or inservice teacher training programs. 

_c) For use in graduate Research, Development, Dissemination or 
Evaluation courses. 

_d) For use in undergraduate RDD&E courses . 

_e) For reference in a library or media center. 



29. Where do you usually listen to the cassettes? 

a) At home. 

b) At tho office. 



_c) In a classroom. 
d ) In a library or media center. 
_e) In a car, bus, plane, etc. 



30. Do you have your own personal cassette tape recorder or player? 



yes 

no 



31. What specific topics would you like to have presented on tape which 
aren't currently available? 



32. What is your primary area of interest? 
a) Research 



_b) Development 
c ) Dissemination 
_d) Evaluation’ 

e ) Instruction 

f ) Management 
_g) Other 



(what ’I 
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33 ‘ tapes?° f thC t0piCS belOW WOUld you most like to see Presented on audio 
_a) "How-to-do- its" (demonstrations of technique) 



_b) Dialogues between prominent advocates with different points 
of view. 

_c) Discussion of current "hot" issues. 

_d) Summaries of current research findings. 



_e) Discussions of recent funding, law, and policy. 






Which medium do you prefer for receiving information? 
jO Convention paper sessions or workshops. 



b ) Video tapes 
_c) Films . 

_d) Audio tapes. 

_e) Books and manuals. 

_f) Journal articles and abstracts 



35- Would you purchase audio-tapes agaih? 



yes ' , 

no 

Don't know 



36. Would you consider paying $30-$l+5 for a six month-tape trial subscription 
for a cassette-based "journal" of education in an area of interest? 



yes 

no 

Don't know 
If yes, what area? 



37. 



How would you rate the speed with which your order was processed by AERA? 
a) Excellent 



b ) Satisfactory 

c ) Unsatisfactory 



Thank you for your cooperation. Please place this questionnaire in the stanroed 
self-addressed envelope provided and mail to - Tape Evaluation ’ 

c /® Lab for Educational. Development 
North Cottage Grove 201-B 
Bloomington, Indiana 47401 
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